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Business Education 

Often the aim in business education is to impart "general principles" to the student. The 
hope is that by teaching the student general principles (whether in Marketing, Finance, 
Organizational Behavior, or Accounting), the student will know how to apply those 
principles in the right circumstances in the future. However, students who are taught 
decontextualized facts and rules n<ay have difficulty remembering and applying what 
they have learned. A number of cognitive and educational theorists have stressed the 
importance of learning problem-solving skills, integrating the teaching of thinking skills 
and content, and situating instruction in meaningful contexts (e.g. Resnick & Klopfer, 
1989; Brown, Collins, and Duguid, 1989; Bransford, Goldman, and Vye, 1991). 

Much of the knowledge in the various functional areas of business is not easily captured 
with facts and rules; instead, knowledge is bound up in the experience of practitioners. In 
view of these circumstances, we believe (in agreement with many graduate schools of 
business administration) that the extensive use of the case method in business education 
is a wiser and more effective approach than the various forms of the "general principles" 
approach. 

Although there is not just one form of the business case method (Dooley and Skinner, 
1977), the "classic" form of the business case method is aimed at promoting decision- 
making skills (Hunt, 1951). Students are given case materials that describe a complex, 
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real-world business situation that has a problem or issue requiring a decision. Before 
class, they analyze the case and prepare a recommended decision or solution. In class, 
which may typically have between 20 and 80 students, the instructor leads a discussion in 
which recommendations from different students are critiqued and debated. Often, 
students also hand in a write-up that is graded by the instructor or an assistant. More than 
one credible solution to the business problem must be possible (McAninch, 1993). Cases 
in business range from short descriptions of relatively simple business problems to highly 
complex narratives that include "internal company information as well as external 
industry data, and psychological, sociological, and anthropological observations as well 
as technical and economic material" (Christensen, 1987). 

A great deal has been written about the merits of the business case method. However, in 
practice the principles and ideals of case method administration are often not reached 
(e.g. Argyris, 1980). And Smith (1987) has pointed out that there has been little research 
that has closely examined the effectiveness of the business case method. Some 
assumptions about the case method are seldom examined. For example, case method 
proponents sometimes point out that not only do students learn to do good analysis in the 
given business domain, but also they learn how to communicate, express and defend their 
ideas, dynamically evaluate other people's arguments, and so forth. The latter is certainly 
a worthy educational objective, but the question of whether the learning of these two 
skills should be conjoined is not addressed. Is it conceivable that students would learn 
analysis skills better in a context other than the class discussion? Is it conceivable that 
students would learn communication and argumentation skills better in some other 
context, e.g. in a context where the students are not struggling with new concepts and 
analytical methods in a relatively unfamiliar domain? Such questions are seldom 
discussed 

Most of the literature takes the current implementation of the business school case 
method as a given. However, at a closer level of scrutiny, several shortcomings of this 
implementation can be discerned. 

First, help for the student while he 1 is working on the case is limited or sometimes 
unavailable. Sometimes office hours with a teaching assistant are an available resource, 
but asking questions this way is often inconvenient, slow, and limited. If a group is doing 
the case assignment, other students can be consulted, but they often lack relevant 
experience and are more prone to making weak or erroneous assertions. Students 
working on their own may run into frustrating impasses which can make the entire case 
analysis task distasteful. 

Second, feedback to the student is limited and comes long after the student performs his 
analysis. Due to the logistics of running a large class, many of the students will not 
typically get to present their own analyses and recommendations in a class discussion, 
and even those who do present have no assurance that their analysis will receive much 
attention in the discussion. When students' write-ups are graded by an instructor or 
assistant, if any comments are written at all they are often short or cryptic. And such 
comments are only seen by the student long after the analysis is performed, when the 
student may have forgotten details of the case and the thoughts that went into their own 
analyses. 

Third, the method depends on the availability of a qualified human instructor who can 
give the students feedback, discussion, or an evaluation of their recommendations. No 
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student will* perform an analysis and write a report if he knows that it will never be read 
or evaluated. 

Fourth, due to time constraints or teacher unwillingness, a concrete, detailed, step-by-step 
solution is rarely demonstrated to the student. Seeing suclvconcrete, detailed solutions is 
invaluable for inexperienced students. However, the instructor realizes that many of the 
steps in any particular solution path may be subject to uncertainty, and may not want 
students to get the impression-even if a solution is flagged as a "possible" solution— that 
only one solution is correct. 

Fifth, students' motivation for analyzing the case may too often be social and "indirect" 
instead of a direct interest in succeeding in the administrative scenario. Instead of a direct 
interest in "participating" in the case's administrative scenario, students' motivations often 
center on getting a good grade, impressing the instructor, and/or protecting their egos 
during the class discussion. Argyris (1980) documents various "games" and 
"camouflaging of games" that occur during the case discussion. 

A learning-by-doing approach implemented in a computer-based environment can be 
effective in addressing each of these shortcomings of the traditional case method. 

FRA (Financial Report Analyst) is an intelligent, multimedia software system that is 
designed to teach topics and skills in Financial Accounting. It is based on an architecture 
called a goal-based scenario. Grounded in cognitive principles of learning and memory, 
a goal-based scenario (GBS) provides a learning-by-doing environment which places the 
student in a specific problem-solving role in a realistic scenario and provides intelligent 
coaching as the task is performed. 

A GBS designed to teach Financial Accounting or other topics in Business 
Administration provides individualized tutoring and coaching, which is typically 
impossible to provide in a large business class. By constructing a realistic task and 
having the student use the data in FRA to make important decisions about committing 
financial resources, the student will see the usefulness of the knowledge being taught and 
consequently is more motivated to learn the subject matter. 



Goal-Based Scenarios 

A goal-based scenario places specific constraints on the selection of material to be taught, 
the goals the student will pursue, the environment in which the student will work, the 
tasks the student will perform, and the resources that are made available to the student 
(Schank, Fano, Jona, and Bell, 1994). The idea of GBS's is grounded in theories of goal- 
based learning-by-doing (Schank 1994b), failure-driven learning (Schank 1982), and 
case-based remediation (Riesbeck and Schank, 1989; Kolodner, 1993). 

In FRA, the student's goal is to analyze a set of financial statements and to make good 
enough recommendations to earn a promotion. Running a trucking company, managing a 
mutual fund, and deciding on a marketing strategy are each examples of other goals that 
might be suitable for a GBS. In FRA, the student plays the role of a junior lending 
analyst at Uptown Bank who must decide whether or not to lend money , to different 
companies. 

In addition to providing an engaging and appropriate role for the student, a GBS must 
satisfy other criteria in order to provide a successful learning experience. The GBS must 
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present both a set of target skills and an environment that will motivate the student and 
enable the productive use of the skills upon completion. The following issues are 
considered in the design of any GBS: 

• thematic coherence - A clear relation between activities and the student's role is 
essential for student motivation. For example, getting a promotion for correctly 
computing a set of ratios would violate the spirit of this requirement. In FRA, the 
student must perform good analyses to avoid non-performing loans and the ire of 
superiors, which is typical of a lending analyst's situation. 

• realism/richness - Realism is critical for ensuring that the indexing of the 
student's experiences into his memory is adequate for retrieval in relevant future 
situations. Solving a problem In the context of a three-line scenario description 
makes future recall difficult because of the lack of richness and vividness in the 
experience. FRA instead has an elaborate, realistic problem scenario which elicits 
authentic problem-solving behavior. 

• control/empowerment - The student must feel responsible for the completion of 
the task and must have great latitude and flexibility in choosing a course of action. 
FRA allows the student to: explore the data in his own way, perform what-if 
analyses, experiment using a general-purpose spreadsheet tool, request coaching, 
and draw conclusions in whatever order he chooses. 

• responsiveness - The student must see the effects of his actions and must have 
immediate access to useful feedback. In FRA, the student can make specific 
lending recommendations and then observe their effects. 

• pedagogical goal support - The scenario must be compatible with and support 
the acquisition of the target skills. FRA embodies a scenario in which the skills of 
financial statement analysis are critical. 

• pedagogical goal resources - Carefully-chosen intervention strategies and 
materials must be available to assist the student. FRA provides context-sensitive 
coaching whenever the student requests it, and can demonstrate expert analysis. 

Several prototype systems in different domains have been built at the Institute for the 
Learning Sciences using the basic GBS framework, including systems for sales training 
(Kass et al., 1994), social studies (Kass and Guralnick, 1991), and medical diagnosis 
(Bell, Bareiss, and Beckwith, 1994). A number of other GBS systems are also in 
development at the Institute. 

Embedding skills to be learned within some target activity is characteristic of most 
apprenticeship styies of learning. "Cognitive apprenticeship" (Collins et al., 1989) adapts 
characteristics of traditional apprenticeship instruction to cognitive processes. The GBS 
framework shares with cognitive apprenticeship an emphasis on the practice of skills 
within an authentic context, with both proposals drawing from prior work in situated 
cognition (Brown et al., 1989). Cognitive apprenticeship is based on the teaching 
methods of: (1) modeling, or demonstration by an expert (like the expert analysis in 
FRA); (2) coaching, which consists of immediate feedback to the student and in-context 
help and advice while the student is engaged in the activity; and (3) scaffolding, or 
provision of tools, suggestions, and/or partial solutions which reduces the cognitive load 
on the student. 
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These ideas are consistent with another approach called "anchored instruction" 
(Bransford et al., 1990). This approach calls for creating an authentic task environment 
in which learners can appreciate the utility of skills and knowledge they are acquiring 
and, furthermore, can recognize conditions under which these skills are applicable. 
Anchored instruction is similar is some respects to the GBS framework: both recognize 
the importance of situating learning in an authentic activity, and both regard the authentic 
practice of skills as essential. Goal-based scenarios and anchored instruction have in 
common the end goal of overcoming the classic "inert-knowledge problem", a 
phenomenon identified by Whitehead (1929) in which knowledge can usually be recalled 
when people are explicitly asked to do so but it is not used spontaneously in problem- 
solving even when it is relevant. 

Studies have been conducted at the Institute for the Learning Sciences to test our 
hypotheses concerning the effectiveness of GBS's. For example, in a study of a goal- 
based scenario that teaches about genetic counseling and Sickle Cell Disease (Bell, 
Bareiss, and Beckwith, 1994), it was found that users of the program learned more about 
the conditions of applicability of the new knowledge, that is, the relevance of basic 
concepts to problem solving. Users were found to reason about Sickle Cell Disease with 
more correct concepts and with fewer misconceptions about relevant information than if 
they had learned from a more traditional medium. 

Little empirical work exists for the evaluation of learning by doing in the area of business 
administration. However, if the inquiry is broadened to include learning by doing in 
other types of professional education, isolated studies can be fo md in the area of 
problem-based learning, which is a case-oriented learning by doinj^ approach used in 
advanced medical education. In one of the most interesting of these studies, Moore-West 
and O'Donnell (1985) found that the clinical performance of students educated with 
problem-based learning was rated as significantly better than students taught with 
conventional methods. 



The Architecture of FRA 

FRA consists of a performance environment and a coach. In this section, the various 
components of the analysis environment will be described, followed by a discussion of 
how coaching supports the student in performing his task. 

The analysis environment 

In FRA, the student plays the role of an analyst in the corporate lending department of a 
bank. The student is given a series of company files containing increasingly complex 
financial statements and other relevant data, each involvirig*u.request for some kind of 
long-term or short-term debt financing. The student's job is to analyze each set of 
statements and to produce a recommendation report which lists his conclusions about the 
company and his recommendation concerning the proposed loan. When the assigned 
reports are completed, a simulated calendar is advanced to the next quarter, and the 
student receives new company files for analysis. After each company analysis, the 
student sees how each company actually performs in subsequent quarters. The student's 
objective is to make good loans and avoid non-performing loans, and so the student's 
performance will be rated over time. During each company analysis, the student is given 
access to extensive on-line help and coaching, as well as access to an embedded 
hypermedia system containing useful text, video, and "mini-demonstrations." 



The student works in an analysis environment which contains: all of the items of the 
current company file; a simple spreadsheet with capabilities similar to (though less 
elaborate than) products like Microsoft Excel (TM); and a button pad (Jona et al., 1991) 
which the student uses to navigate through FRA and to ask for information or coaching. 
See Figure 1. 

The company file contains the financial statements, the financial statement footnotes, 
supplementary information, and text documents typically including: a "summary 
memorandum" that provides an overview of the situation; a "company information" 
document describing some of the qualitative factors and issues involved; and a "loan 
request" describing what kind of loan is being contemplated. The student can easily flip 
though the items in the company file and use data in the financial statements to perform 
trend analyses, ratio analyses, projections, consistency checks, and other calculations in 
his spreadsheet. 
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Figure 1 . The analysis environment. On the left side is the company file and the company file item 
selector, which in this case contains eleven items. On the right is a general purpose spreadsheet, and at the 

bottom is the button pad. 



The recommendation report 

When the student clicks on a button labeled "Recommendation Report", the report form is 
displayed in the analysis environment. 

The student's objective in each case is to complete a report that details his specific 
conclusions about the company and makes a recommendation concerning the proposed 
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loan. The recommendation report serves several important purposes. One is to focus the 
student's attention on what the specific goals of the analysis are and to make him aware 
of what issues are relevant for the task and domain. A second is to get the student to "go 
out on a limb" and make a specific diagnosis and recommendation. Third, the report 
serves as a limited but . needed window into the student's mind: as the student draws 
specific conclusions and so indicates on the recommendation report, the program gauges 
how well the student is grasping the significant issues in the case and what problems or 
misconceptions he might have. 

The recommendation report begins by asking for a number of broad financial 
characteristics such as "overall financial soundness", "accounting practices", and 
"liquidity." The student indicates his assessment for these categories, and, based on his 
response, is often directed to specify particular supporting findings coming out of his 
analysis. Thus, some parts of the recommendation report behave like hierarchical menus: 
the high-level conclusions are indicated first, and then the more detailed conclusions 
which support the high-level conclusions are indicated. The format of the 
recommendation report is designed to structure the argument being constructed by the 
student; it reflects an expert model of financial statement analysis, and thus provides 
scaffolding in the cognitive apprenticeship sense discussed previously (cf. Collins & 
Ferguson 1993). Figure 2 shows an example of a completed recommendation report. 



Recommendation Report 

Company: Th« Ftagina Conpany 
Report by: Jean Kno»a!ot 



■ BANK 
Date: September 1, 1988 



CONCLUSIONS 
Ouerall financial soundness: 



Fair 



Financial Problem Areas (check one or both): □ Balance sheet 
Particular Problem Areas (check all which apply): 

□ Buildup of i nventorles ^ Overstated accounts receivable 

□ Stretching trade credit □ Slow or decreasing sales 
Q Unprofitable operations £3 Unacceptably high costs 



§3 Cash flows and/or operations 



[3 Other (explal n) : [poor co^trag* ratio 



[3 Questionable or improper accounting decisions 

□ Earnings manipulations 

[3 Questionable estimates Describe: | r«c* i vab I *s allowance shrinking too fast 

□ Material Information unreported 

□ Misclasslfied accounts or transactions 
Unfavorable auditor's opinion 

Predictions for duration of loan period (assuming loan is made as proposed in Loan Request memo) 
Profitability: [Fair 
Liquidity: 
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Predicted Net income in year of maturity: $| 



[Poor 



Likelihood of ability to make all agreed payments to Uptown Bank as scheduled: | Medium 
Likelihood of bankruptcy/insolvency: [Low V| 

Other conclusions 
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Overall accounting practice innnMninn — i 



LOAN RECOMMENDATION: | Lend only with special terms or covenants ^ | 



Terms or couenonts to demand: 

Precautionary provisions 
□ Collateral 
[2 Divide. iu restrictions 

^ Call-in provisions Condition(s) of technical default: 
If 



or if 

Speculative provisions 



deb t- to-equ i ty ratio 



9<*s | above y [ value: 
soea I above ▼ | value: 



5C| 



[ Resume analysis, tuork on report later j [ Recommendations complete: Submit report ) 



Figure 2. A recommendation report that has been completed. The top portion indicates the student's 
conclusions and some of their supporting justifications and evidence, and the bottom portion indicates the 

student's specific recommendation. 



Expert analysis 

After the student has submitted a recommendation, the student finds out how the 
company subsequently performed. He then has the opportunity to see a demonstration of 
how an expert analyzed the financial statements of the company. The expert analysis is 
performed in the same analysis environment as that used by the student in his own 
analyses, i.e. with the items in the company file, the spreadsheet, the recommendation 
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report, etc. Step by step, an "invisible expert" types formulas into the spreadsheets, flips 
back and forth to the different financial statements, and jots notes onto the findings 
scratchpad. Each step is annotated with either an audio commentary or a text description 
of what the expert analyst is thinking: what his current action is, why he is doing it, the 
hypotheses or conjectures he is pursuing, and/or the conclusions he is drawing. The 
student, having studied the same case in the same analysis environment, gets a clear and 
concrete picture of the sorts of things one must do during an analysis, as well as a 
valuable point of reference on the actions and conjectures the student tried during his own 
analysis. (The expert analysis is, of course, not the only "correct" analysis, and the 
student is made aware of this.) 

The progress of the expert analysis is controlled by the student. At any time, he can stop 
it to ask questions via the hypermedia system, to retrieve his own previous analysis for 
comparison, or to quit the expert analysis. At the present time, a deterministic path is 
used for the steps in the expert analysis sequence, although there is no theoretical reason 
why the expert analysis cannot have branching points based on the student's initial 
recommendation report or on dynamic interactions with the student during the expert 
analysis. 



Coaching 

Coaching in FRA is similar to the situation where an apprentice has been working on an 
involved, complex activity with a human mentor nearby, and then realizes that he wants 
help and calls the mentor over. Sometimes, with a quick look at the apprentice's 
progress, the mentor can give a terse piece of advice which immediately satisfies the 
apprentice. On the other hand, often the apprentice has some responsibility for 
communicating to the mentor what his specific problem is, since there are a large number 
of paths the apprentice could have taken, and the mentor does not have a detailed model 
of the apprentice's mental state. After the mentor has selected and delivered the most 
appropriate piece of advice, the apprentice often will have follow-up questions. The 
apprentice may want examples and stories which will give him perspective or a referent 
to concrete experience; he may want clarifications and explanations; he may want more 
detail, or perhaps the "big picture"; he may want a demonstration of a technique. He will 
often want to ask several questions in succession, perhaps sometimes backing up to 
previous topics. Eventually, the apprentice will be ready to return to his task. 

The mentor, knowing well the ins and outs of the problem that the apprentice is working 
on in this instructional setting, will sometimes notice an aspect which the apprentice has 
overlooked, or some technique which it would benefit the apprentice to try. The mentor 
will seldom interrupt immediately, but instead will wait to see if the apprentice figures it 
out himself, because he knows the satisfaction the apprentice will get if he does figure it 
out by himself, and realizes that he will learn it more effectively that way. However, at 
some point the mentor will make the apprentice aware that he has a piece of advice if the 
apprentice is interested; if the apprentice assents, the advice is given and then follow-up 
questions are answered. 

FRA is designed to have all of the basic coaching capabilities that this mentor has. 
Coaching rules determine which advice is appropriate in a given circumstance. The 
conditions part of each rule consists of combinations of tests that are implemented using a 
relatively small library of testing functions. Testing functions either test the current state 
of the analysis environment, or query a dynamically-maintained record of all events and 
actions which the student has performed so far. Coaching is thus context-sensitive: 
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exactly what kind of coaching is given at a particular time is dependent on what actions 
the student has done so far, what conclusions he has drawn, and what advice has already 
been given. 

Students are coached while they are performing their analyses. To provide students with 
memorable perspectives on the task of financial statement analysis, we have gathered 
videotaped interviews with an accounting professor at the Kellogg Graduate School of 
Management at Northwestern University and with a number of experienced analysts in 
the banking industry. Upon the student's request, brief, contextually-relevant video clips 
containing war stories and analysis techniques are presented to the student while he is 
working in the analysis environment (Figure 3). The cases which the student analyzes 
provide a context in which the relevance and the usefulness of the experts' words 
becomes clear and memorable; the student cares about the information being given 
because it will help him to draw the right conclusions and make the right 
recommendation in his own case assignment. 

Two interface functions support coaching. One of the buttons on the button pad is 
labeled "Now What?". Depending on the situation, when the student clicks on this button 
he can receive hints or suggestions, be reminded of his current goals, see a "mini- 
demonstration" of a technique, or engage in an interactive dialogue. Whenever the 
student receives advice or information, he can ask follow-up questions via a hierarchical 
"Ask menu" that contains hypermedia links to other relevant pieces of information 
(Ferguson et al., 1992). 




Figure 3. Here, advice has been requested by the student, and in response a video clip of a lending expert is 

provided. 
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In addition, a button labeled "New Advice*' provides a mechanism for tutor-initiated hints 
and suggestions. While the student works on the case, the tutor "watches" in the 
background and observes the student's progress; if the coaching rules suggest that the 
student needs a particular bit of advice, and the student has not been asking for help, the 
"New Advice" button is enabled and illuminated. However, the student remains in 
control. If and when the student clicks on this button (in our experience, the student 
always does), the advice is presented. 



Usability Test 

When building interactive computer artifacts, usability is a serious and complex issue. 
Often, the program designers believe the initial versions of their software are quite 
sensible, well-planned, and neatly organized, but when it is placed in front of target users, 
the users may be bewildered and may have any of a number of problems. They may be 
confused about the function of parts of the interface; they may fail to recognize available 
resources; or they may fail to comprehend the overall organization of the available 
information or the structure of the program itself. 

In the case of educational software, usability is especially important. In addition to the 
fact that many users may not be sophisticated about using computers, users are trying to 
learn unfamiliar concepts or material, which may diminish their patience or capacity for 
figuring out the interface. And, unlike the case where a user is willing to invest effort in 
learning how to use a new productivity tool, the normal expectation is that an educational 
program will only be used for a limited period. 

At the same time, tests of highly interactive educational software present unique 
complications. When the environment is sufficiently rich, different students will have 
different experiences, will explore different areas, and will follow different paths of 
action; this complicates the aim of a "controlled experiment." Furthermore, usability may 
be difficult tc infer from examining the interactions only. In light of this, it is important 
to include the user's own evaluation of the software and of their experience with it as part 
of the usability test. 

A usability test of FRA was conducted in February 1994 with a class of MBA students at 
the Kellogg Graduate School of Management at Northwestern University. 21 students 
used FRA as a required assignment in course entitled Corporate Reporting and Analysis, 
taught by Professor K. Ramesh. All students had previously taken an introductory course 
in Financial Accounting. 

In their interaction with FRA, students analyzed a detailed case involving a medium-sized 
manufacturer; the case highlighted issues involving inventory and accounts receivable 
accounting, cash flow analysis, discretionary accruals, and reporting fraud. The content 
of the case fit strategically into the course curriculum. The case was fairly complex; the 
company file contained eleven separate documents, six of which were number-filled 
financial statements or summaries. The goal of the test was to expose any problems with 
the user interface, to see whether the problem scenario and analysis environment were 
comprehensible to students, and to obtain students 1 reactions to the program. Due to 
scheduling constraints, most students were not allowed to work on FRA more than four 
hours; almost all of the students maue use of the full allotment of their time using the 
program (including case analysis, submission of a recommendation report, and 
demonstration of the expert analysis.) 
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Three types of data were collected: (1) while the program was used, all student actions 
were automatically recorded and saved for later review; (2) all students filled out a 
questionnaire after use; and (3) students were informally interviewed during and after 
use. 



Overall, the results were very positive. Students generally found FRA to be 
understandable, helpful, and "user friendly", and uniformly claimed to enjoy using it. As 
one student wrote, "Interactive makes it more interesting. Immediate feedback and 
guidance is helpful." With respect to the coaching, different students judged it as being 
"Excellent", "Great help", and that it "Helped keep my thoughts focused." 

Several of them singled out the expert analysis as being particularly effective;' one student 
stated, "This was the most helpful aspect of the program. I gained a ton of insight into 
how an analyst thinks." However, a number of the students complained that the 
spreadsheets lacked features that they expected and wanted. And many of the students 
wished they could have had more time to explore the system (which was not possible 
during this test due to the scheduling constraints). 

Table 1 lists selected summaries of the students' activities. 



Total analysis time, in minutes (from start to time of report submission, excluding 
clock pauses): 

Least 126 
Most 239 
Average 193 



Number of worksheet cell entries by students: 



Least 81 
Most 249 
Average 1 49 



Table 1 
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As shown, the total analysis time averaged three hours and thirteen minutes; this amount 
of time does not include clock pauses (initiated when, for example, the student 
temporarily left the room), and it does not include viewing of the expert analysis, which 
took roughly one half hour on average. The shortest amount of time was taken by a 
student who currently works in the Corporate Finance Division of a Chicago bank; he 
very quickly came to a "Do Not Lend" decision, and admitted afterwards that the decision 
was not especially difficult for him. He also opined that the program was "a little 
generous in the information and hints provided." 

As is also shown in Table 1, all students used the worksheets quite actively. This 
suggests that the worksheet is an important resource to provide for the student to "think 
through" the analysis, or at least that the students believed that such activities was useful. 
Note that, although FRA's architecture does not prevent doing so, the program did not 
make any effort to interpret or reason about the students worksheet entries. Fortunately, 
all of the students had some experience with commercial spreadsheets. This familiarity 
of course cannot always be presumed; while FRA provided basic online reference and 
help for the worksheets, most students did not avail themselves of this help. 
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In the tests, we saw that the first two or three users of FRA were spending an inordinate 
amount of time calculating financial ratios; consequently, after five students had used 
FRA, we inserted an additional document into the company file that pre-computed some 
basic ratios for the student. Professor Ramesh judged that this was a mechanical detail 
that would not significantly reduce the difficulty of the problem. Not surprisingly, some 
of the first five students were at the low end for number of dialogues and video clips seen, 
presumably because they were too occupied with calculations. 



Table 2 summarizes information related to the coaching in FRA. 



Total number of "dialogues" seen by student during analysis (before report submittal): 


Least 


29 


Most 


144 


Average 


91 


Total number of video clips viewed: 




Least 


4 


Most 


25 


Average 


16 



Table 2 



A "dialogue", as defined in FRA, is any piece of advice or information that is presented to 
the student in the center of the screen during the course of the interaction; a dialogue may 
be a video clip, a short piece of text (typically no more that two paragraphs), or a small 
spreadsheet demonstration. As shown, there was wide variation in the amount of 
information obtained by students. The same can be said for the number of video clips 
viewed. Video clips generally are between one and four minutes in duration. The highest 
number of video clips seen was 25; recall that FRA includes over one hundred video clips 
(none of which are specifically tied to the particular case that the students analyzed). 
Note also that the total number of dialogues in the system (including the videos) is close 
to five hundred. 



Final recommendation: 




"Lend, without special conditions" 


0 students 


"Lend only with special terms or covenants" 


1 1 students 


"Do not lend - too risky" 


10 students 



Table 3 



As shown in Table 3, none of the students recommended lending to the company without 
special conditions. This is encouraging because, in this particular case, the company was 
fraudulently hiding unfavorable information and funding large losses with increased debt; 
however, detecting this was no easy matter. This suggests that, at least to some degree, 
the coaching "worked." While the coaching was designed not to "give away" the 
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"correct" analysis, it was also designed to keep students from getting lost or missing key 
issues in the case; and apparently none of the students did. The recommendation reports 
completed by students were generally judged as being sensible and not too far off from 
the "intended" analysis. 

A few other interesting findings were obtained from a closer examination of the recorded 
action-transcripts. 

Almost all of the students used all of the available resources in the analysis environment: 
the Ask menus, the worksheets, the NowWhat? button, the New Advice button, the 
Terms/Topics button, and a "hypertext" facility for getting advice about parts of the 
recommendation report. This indicates that students did not have too much trouble 
figuring out how to use the interface. 

Recall that the NowWhat? button is the device that the student uses to request coaching 
and advice. One of the coaching rules specified that if, in the first forty minutes of 
interaction, the student clicked the NowWhat? button less than three times, the suggestion 
would be offered to the student that they should try using it more often. Interestingly, 12 
of the 21 student received this advice. This suggests that, without such explicit advice, 
some students are unlikely to use the NowWhat? button as often as we would like them 
to. 

On the other hand, the behavior of a few of the students suggested that they may have 
been trying to "pump" the NowWhat? button repeatedly, presumably so that they could 
get more hints and advice. Fortunately this kind of situation is handled in FRA with a 
mechanism that enforces a delay in the availability of some kinds of advice; if a student 
clicks NowWhat? too soon after getting advice from a prior NowWhat? click, then, 
instead of specific advice the student is usually presented with a list of general topics 
about lending and financial statement analysis, and the student must interactively indicate 
what kind of information is desired. 

Exploration of the hypermedia system was more limited than what had previously been 
hoped, and this may indeed have been due to the artificial time constraints. Most students 
praised the video clips, and one suggested that there should be an index of available 
videos. 

In general, the tests indicate that FRA's architecture is viable, which encourages our 
belief that refinements of the architecture will make it a practical system for presenting a 
wide range of case problems in Financial Accounting and other areas of business 
administration. 

At the same time, these tests do not indicate the pedagogical effectiveness of FRA. In the 
future, we would like to conduct tests that afford some such indication. To do this, we 
must design appropriate transfer tasks, measure short- and long-term retention, and, of 
course, compare the performance to a control group which is taught the same information 
by traditional means. 

As was mentioned previously, in this kind of system the student's actions are not closely 
constrained. Different students will have different experiences, will explore different 
areas, and will follow different paths of action. In our tests, for example, we found that 
different students had different calculations on their worksheets, looked at the documents 
in different orders and for different durations, used the Ask menu to different degrees, 
and did not view all of the same dialogues. With a program like FRA, it is therefore 
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more difficult to assign credit or blame for what the student does or does not learn during 
the interaction, and this can make evaluation an interesting challenge. 



Conclusion 

In the introduction, we pointed out five shortcomings in the business case method which 
can potentially be addressed by a system like FRA. FRA provides coaching and advice 
while the student is performing his analysis. It provides students with immediate and 
relevant feedback: they receive suggestions concerning their selections on the 
recommendation report, they find out about the company's subsequent performance, and 
they can compare their own analysis to an expert's. FRA does not require the availability 
^f a human instructor. It provides concrete, step-by-step demonstrations of expert 
analysis methods. And it can increase the student's direct interest in the scenario. 

We also mentioned a more general question, namely, whether the learning of 
communication skills and argumentation skills should be conjoined with the learning of 
analytical skills. (Once again,) is it conceivable that students would learn analysis skills 
better in a context other than the class discussion? 

Both in the literature about the case method and in practice, much of the focus is on the 
classroom discussion rather than the work the student does when performing his analysis. 
We suspect that this focus may have the effect of "de legitimizing" the student's sense - 
making and solution-finding activities. Not only are students not given support or 
coaching while they are performing their analytical activities, but these activities are 
often not acknowledged or demonstrated in the climactic class discussion. The result 
may be less motivation to perform important but painstaking detail work. A detailed 
multivariate sensitivity analysis that may not be easy to describe in the class discussion 
may be jettisoned in favor of a radical, attention-getting theory whose plausibility is then 
rationalized with limited case data. Also, when a student discovers that he is going down 
a blind alley during his own analysis, he is likely to view it as a failure rather than as a 
useful experience. 

In contrast, FRA is completely focused on the student's analytic and problem-solving 
activities. Furthermore, unlike class discussions in the traditional case method, in which 
only some of the students will present their work and interact with the instructor, FRA 
provides individualized instiuction to every student. Thus FRA exemplifies a promising 
alternative to the case method as traditionally implemented in business schools. 
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